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Abstract 

This study was designed to examine the impact NCLB defined teacher-quality variables (1) 
major/minor in mathematics, (2) highest academic degree, (3) type of teaching certificate, (4) 
years taught mathematics (NCLB, 2001) on the mathematics average scale scores of eighth- 
grade students. The study used a secondary analysis of the 2007 National Assessment of 
Educational Progress (NAEP). Results were reported in terms of statistical significance ip. <01) 
and effect size (Cohen's d). This study found that an eighth-grade mathematics teacher is more 
effective with (1) either a major or minor in mathematics, (2) a Professional degree, (3) a 
Regular/standard teaching certificate, and (4) with 20+ years of teaching mathematics 
experience. The teacher quality variables had an impact on the average scale scores of students 
who took the 2007 NAEP eighth-grade mathematics test (national data used in the secondary 
analysis). 
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Is Teacher Quality Related to Eighth-Grade Mathematics Achievement? 

Evidence from the 2007 NAEP Data 

The No Child Left Behind (NCLB) Act (2001) included "teacher quality" as a major 
factor that is likely to affect student learning. Teacher quality was defined by NCLB in terms of 
teacher credentials. For mathematics teachers, these were: (1) major/minor in mathematics, (2) 
highest academic degree, (3) type of teaching certificate, (4) years taught mathematics (NCLB, 
2001). This manuscript explores the impact of these defined variables on the average 
mathematics scale scores of eighth-grade students measured by the 2007 National Assessment of 
Educational Progress (NAEP). The national dataset was used for this secondary analysis. 

Review of the Literature 

Researchers recently reported that the quality of the teacher in the classroom was the 
most important predictor of student achievement (e.g., Darling-Hammond, 2000; Darling- 
Hammond & Youngs, 2002; Hanhushek, Kain, & Rivkin, 1998).Wayne and Youngs (2003) 
found strong links between the NCLB-defined teacher quality variables and student achievement. 

Papanastasiou (2008) examined the Trends in International Mathematics and Science 
Study (TIMSS) data and found that teaching was the factor that accounted for the greatest 
differences between more effective and less effective schools were teaching. Clotfelter, Ladd, 
and Vigdor (2007) performed a longitudinal analysis of a 10-year administrative dataset from 
North Carolina. They concluded that a teacher's (1) experience, (2) test scores and (3) regular 
licensure all had positive impacts on student achievement; The impact was larger on math 
achievement than on reading achievement. The three teacher measures combined had larger 
effects on mathematics achievement than either class size or the socio-economic characteristics 



of students. 
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Klecker (2007) found statistically significant (p. <01) differences, with moderate 
effect sizes, in the eighth-grade mathematics achievement by teacher-quality measures. 

However, a later analysis of Kentucky fourth-grade reading scores by teacher quality 
variables found no differences in scores by teacher quality (Klecker, 2008). 

Method 

Data Source 

The National Assessment of Educational Progress (NAEP) has since 1969, been the only 
nationally representative and continuing assessment of what America's students know in various 
subject areas. The No Child Left Behind Act (2002) required participation of all school receiving 
Title I money from federal funds. This requirement has enlarged the data base. In 2005, teacher 
questionnaire data were added to include the NCLB -defined teacher quality variables. These 
teacher-quality variables were retained in the 2007 NAEP data (NCES, 2008). 

Data Analysis 

The researcher attended training to analyze the unique NAEP data in June 2007. During 
this visit, the data analyses of NAEP of 2005 data were performed. For the follow-up of the 
secondary analysis of the 2007 NAEP scores, the researcher used —as advised by NAEP training 
staff— the NAEP Data Tool available on the NAEP website. [Note: IRB approval was obtained 
for this study.] 

Results 

Tables 1 and 2 presents the results of the analysis of average scale scores of national 
eighth-grade mathematics scores by the first NCLB teacher quality variable "major/minor in 
mathematics." Table 1 presents the descriptive statistics and Table 2 presents the results of a 



regression analysis. 
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Table 1 

Average Scale Scores in Eighth-Grade Mathematics by Teacher Undergraduate Major/Minor in 
Mathematics 



Category 


Average Scale Score 


SD 


Major 


287 


35 


Minor/emphasis 


284 


35 


Neither 


278 


36 



Table 2 

Statistically Significant (p. <.01 ) Differences in Eighth-Grade Average Scale Scores by 
Undergraduate Major/Minor in Mathematics 



Major Minor/emphasis Neither 



Major 


— 




p. =0.000 


rf=0.09 


p. =0.000 


d= 0.25 


Minor/emphasis 


p. =0.000 


d= 0.09 


— 




p. =0.000 


rf=0.17 


Neither 


p. =0.000 


d= 0.25 


p. =0.000 


d= 0.27 


— 





The analysis presented in Table 2 indicates that there is a small effect (<:/=(). 09) in the test 
scores reflecting the difference between the students of the teacher holding a major in 
mathematics and the students of the teacher holding a minor in mathematics. However, the effect 
on student scores from a mathematics teacher holding either a major (r/=0.25) or a minor 
(c/=0.27) in mathematics is greater than having neither. Thus, a mathematics teacher is more 
effective with either a major or minor in mathematics. 
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Tables 3 and 4 below present the descriptive statistics and inferential statistics respectively for 
the NCLB teacher quality variable "highest academic degree." 

Table 3 

Average Scale Scores in Eighth-Grade Mathematics by Teacher's Highest Academic Degree 



Category 


Average Scale Score 


SD 


High School Diploma 


271 


28 


Associate/Vocational Cert. 


252 


39 


Bachelor's Degree 


280 


35 


Master's Degree 


285 


36 


Educational Specialist 


280 


37 


Doctorate 


279 


39 


Professional 


300 


41 



Table 4 

Statistically Significant (p. =01 ) Differences in Eighth-Grade Average Scale Scores by Highest 
Academic Degree 



Professional Degree 


Associate/Vocational Certification 


p. =0.0031 


Cohen's d = 1.20 


Master's Degree 


Bachelor's Degree 


p. =0.0000 


Cohen's d = 0.14 


Master's Degree 


Educational Specialist 


p=0.0003 


Cohen's d = 0.14 
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There were only three statistically significant (p. <01) differences between categories of 
this variable (Table 4). The large effect size (d= 1 . 20 ) between the average scale scores of 
students whose teachers had an Associates degree or vocational certification and the average 
scale scores of students whose teachers had a Professional degree is remarkable. It is also 
remarkable that there were no other statistically significant differences between the other 
categories. 

Tables 5 and 6 present the descriptive statistics and inferential statistics respectively for 
the NCLB teacher quality variable "type of teaching certificate" and the average scale score in 
eighth- grade mathematics. 

Table 5 

Average Scale Scores in Eighth-Grade Mathematics by Type of Teaching Certificate 



Category 


Average Scale Score 


SD 


Regular/standard 


284 


35 


Probationary 


278 


35 


Provisional 


272 


35 


Temporary 


266 


35 


Emergency 


258 


36 



No Certificate 



284 



37 
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Table 6 

Statistically Significant (p. =01) Differences in Eighth-Grade Average Scale Scores by Type of 
Teaching Certificate 



Regular/standard 


Probationary 


p. = 0.0000 


rf=0.17 


Regular/standard 


Provisional 


p. = 0.0000 


d= 0.34 


Regular/standard 


Temporary 


p. = 0.0000 


d= 0.51 


Regular/standard 


Emergency 


p. = 0.0000 


d= 0.73 


Probationary 


Provisional 


p. = 0.0000 


d= 0.17 


Probationary 


Temporary 


p. = 0.0000 


d= 0.34 


Probationary 


Emergency 


p. = 0.0000 


d= 0.57 


Provisional 


Temporary 


p. = 0.0000 


d=0A7 


Provisional 


Emergency 


p. = 0.0000 


d= 0.39 


Temporary 


Emergency 


p. = 0.0000 


d= 0.23 


No Certificate 


Provisional 


p. = 0.0000 


d= 0.33 


No Certificate 


Temporary 


p. = 0.0000 


rf=0.50 


No Certificate 


Emergency 


p. = 0.0000 


d= 0.71 



The effect sizes for the differences between the average scale scores across categories 
range from small (0.17 - 0.23) to medium (0.34 - 0.51) to large (0.73) (Cohen, 2001). It is a 
surprising finding that the eighth-grade mathematics students of teachers with no teaching 



certification have statistically significantly higher average mean scores with moderate to high 
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effect sizes when compared with the students of teachers with Provisional, Temporary, and 
Emergency Teaching Certificates. 

Table 7 and 8 below present descriptive and inferential statistics for differences in 
average scale scores in eighth-grade mathematics and the NCLB teacher quality variable "years 
taught math." 

Table 7. 

Average Scale Scores in Eighth-Grade Mathematics by Years Taught Mathematics 



Category 


Average Scale Score 


SD 


0-4 Years 


276 


35 


5-9 Years 


282 


35 


10-19 Years 


285 


35 


20+ Years 


289 


35 



Table 8. 

Statistically Significant (p. =01) Differences in Eighth-Grade Average Scale Scores by Type of 
Teaching Certificate 



20+ Years 


10-19 Years 


p. =0.000 


d= 0.11 


20+ Years 


5-9 Years 


p. =0.000 


d= 0.20 


20+ Years 


0-4 Years 


p. =0.000 


d= 0.37 


10-19 Years 


5-9 Years 


p. =0.000 


d= 0.09 


10-19 Years 


0-4 Years 


p. =0.000 


d= 0.26 


5-9 Years 


0-4 Years 


p. =0.000 


d= 0.17 
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The effect sizes range from small (0.11-0.26) to moderate (0.37) (Table 8). The effect 
sizes increase as the years of teaching experience increase. In the NAEP database, years-of- 
teaching-experience is available as a continuous variable. The categories presented are the 
categories available through the NAEP Data Tool. 

Conclusions 

This secondary analysis of the NAEP 2007 eighth-grade mathematics national dataset 
average scale scores found statistically significant (p.<01) differences by NCLB-defined teacher- 
quality variables. Because of the large N in the dataset, the best measure of real differences is 
Cohen's d. Even small effect sizes indicate areas for future research (Cohen, 2001). 
Undergraduate Major or Minor in Mathematics 

The analysis found a small effect for the NCLB variable "major or minor in 
mathematics." The effect on student scores of a mathematics teacher holding either a major 
(c/=0.25) or a minor (<7=0.27) in mathematics is greater than having neither. However, the effect 
size between having a major and having a minor is very small (d=. 09). Thus, an eighth-grade 
mathematics teacher is more effective with either a major or minor in mathematics. 

Teacher's Highest Academic Degree 

This analysis yielded rather puzzling results (Table 4). The only statistically significant 
{p. <01) differences were between Professional Degree and Associate Degree/Vocational 
Certification, with a large effect size (d=1.20). The difference between the scores of the students 
being taught by a teacher with a Master's or a Bachelor's Degree was statistically significant, yet 
with a small effect size (<r/=0. 14). The same statistical significance and effect size was found for 
the comparison of student scores of a teacher holding a Master's Degree and a teacher holding an 



Educational Specialist Degree. 
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Type of Teaching Certificate 

There were many statistically significant differences between the six categories in this 
variable. The average scale score was highest for the students of teachers holding either the 
Regular/Standard Certificate (M=284, SD=35) or No Certificate (M=284, SD=35). The lowest 
average scale score (M=266; SD=35) was for the students of mathematics teachers holding a 
Temporary certificate. The largest effect size (d=0.73) was the effect of the teacher holding 
Regular/Standard certification rather than Emergency Certification. The puzzle of the high 
average scale score for students of teachers holding No Certification calls for further exploration 
of the data and the variable definitions. 

Years Taught Mathematics 

The eighth-grade students who were taught by teachers with the most (20+) years-of- 
teaching experience has the highest average scale score. The largest effect size (d= 0.37) was 
between the average scale score of students of teachers with 20+ years-of-teaching-experience 
(289) and the average scale score of students of teachers with 0-4 years-of-teaching-experience 
(276). The analysis of these data found that the students' scores increased with their teachers' 



years-of-teaching experience. 
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